Skip to content

Standardize Resource Labels #351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Jan 28, 2023
Merged

Standardize Resource Labels #351

merged 17 commits into from
Jan 28, 2023

Conversation

@cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Jan 27, 2023 •
edited
Loading

Closes #273

This updates the labels and the label selectors to follow the recommendations in:
https://helm.sh/docs/chart_best_practices/labels/#standard-labels

Review templates/_helpers.tpl first to see how the labels have changed.

All of the reset of the changes support this, including updates to unit tests and documentation.

This is a breaking change, but: (a) we are on a 0-version, and (b) I wrote a migration script.

The migrations/standardize-labels.sh should add labels to existing resources so that
helm can update them cleanly.

Commits:

  • Use standard k8s+helm labels
  • add app.kubernetes.io/version label
  • update configmap and secret label tests
  • add migrations/standardize-labels.sh

TODO:

  • Test the migration script on an active cluster.

@cognifloyd cognifloyd added this to the v1.0.0 milestone Jan 27, 2023
@cognifloyd cognifloyd self-assigned this Jan 27, 2023
@pull-request-size pull-request-size bot added the size/L PR that changes 100-499 lines. Requires some effort to review. label Jan 27, 2023
mamercad
mamercad reviewed Jan 27, 2023
View reviewed changes
@mamercad
Copy link
Contributor

mamercad commented Jan 27, 2023 •
edited
Loading

Running migrations/standardize-labels.sh:

Output 
❯ RELEASE_NAME=stackstorm-ha NAMESPACE=stackstorm migrations/standardize-labels.sh
configmap/stackstorm-ha-redis labeled
configmap/stackstorm-ha-redis-health labeled
configmap/stackstorm-ha-redis-scripts labeled
configmap/stackstorm-ha-st2-config labeled
configmap/stackstorm-ha-st2-pack-configs labeled
configmap/stackstorm-ha-st2-urls labeled
configmap/stackstorm-ha-st2actionrunner-post-start-script labeled
configmap/stackstorm-ha-st2client-post-start-script labeled
configmap/stackstorm-ha-st2tests labeled
secret/stackstorm-ha-rabbitmq-definitions labeled
secret/stackstorm-ha-st2-apikeys labeled
secret/stackstorm-ha-st2-auth labeled
secret/stackstorm-ha-st2-datastore-crypto-key labeled
secret/stackstorm-ha-st2-kv labeled
secret/stackstorm-ha-st2-ssh labeled
service/stackstorm-ha-redis labeled
service/stackstorm-ha-redis-headless labeled
service/stackstorm-ha-st2api labeled
service/stackstorm-ha-st2auth labeled
service/stackstorm-ha-st2stream labeled
service/stackstorm-ha-st2web labeled
serviceaccount/stackstorm-ha labeled
deployment.apps/stackstorm-ha-st2actionrunner labeled
deployment.apps/stackstorm-ha-st2api labeled
deployment.apps/stackstorm-ha-st2auth labeled
deployment.apps/stackstorm-ha-st2client labeled
deployment.apps/stackstorm-ha-st2garbagecollector labeled
deployment.apps/stackstorm-ha-st2notifier labeled
deployment.apps/stackstorm-ha-st2rulesengine labeled
deployment.apps/stackstorm-ha-st2scheduler labeled
deployment.apps/stackstorm-ha-st2sensorcontainer labeled
deployment.apps/stackstorm-ha-st2stream labeled
deployment.apps/stackstorm-ha-st2timersengine labeled
deployment.apps/stackstorm-ha-st2web labeled
deployment.apps/stackstorm-ha-st2workflowengine labeled
replicaset.apps/stackstorm-ha-st2actionrunner-844d5d96b9 labeled
replicaset.apps/stackstorm-ha-st2api-5499986674 labeled
replicaset.apps/stackstorm-ha-st2auth-7dfc9fdb8f labeled
replicaset.apps/stackstorm-ha-st2client-5bf8b694c8 labeled
replicaset.apps/stackstorm-ha-st2garbagecollector-5f5964574d labeled
replicaset.apps/stackstorm-ha-st2notifier-5dc79694f7 labeled
replicaset.apps/stackstorm-ha-st2rulesengine-6c57fbf4bd labeled
replicaset.apps/stackstorm-ha-st2scheduler-554b9d7496 labeled
replicaset.apps/stackstorm-ha-st2sensorcontainer-556d897b8c labeled
replicaset.apps/stackstorm-ha-st2stream-5bbd79dcd8 labeled
replicaset.apps/stackstorm-ha-st2timersengine-79cd5b5c69 labeled
replicaset.apps/stackstorm-ha-st2web-7c96c574ff labeled
replicaset.apps/stackstorm-ha-st2workflowengine-54dfb489b7 labeled
pod/stackstorm-ha-job-st2-apikey-load-d9w48 labeled
pod/stackstorm-ha-job-st2-key-load-bw8p7 labeled
pod/stackstorm-ha-job-st2-register-content-v2wwx labeled
pod/stackstorm-ha-redis-node-0 labeled
pod/stackstorm-ha-redis-node-1 labeled
pod/stackstorm-ha-redis-node-2 labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-c7294 labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-czjff labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-nzzls labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-tmw2m labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-w8c6x labeled
pod/stackstorm-ha-st2api-5499986674-5kpgl labeled
pod/stackstorm-ha-st2api-5499986674-wqwrc labeled
pod/stackstorm-ha-st2auth-7dfc9fdb8f-mw5fp labeled
pod/stackstorm-ha-st2auth-7dfc9fdb8f-n8cld labeled
pod/stackstorm-ha-st2client-5bf8b694c8-bmfb7 labeled
pod/stackstorm-ha-st2garbagecollector-5f5964574d-ksh8x labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-fnqkq labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-p7x4z labeled
pod/stackstorm-ha-st2rulesengine-6c57fbf4bd-29678 labeled
pod/stackstorm-ha-st2rulesengine-6c57fbf4bd-6z8xh labeled
pod/stackstorm-ha-st2scheduler-554b9d7496-4fzg5 labeled
pod/stackstorm-ha-st2scheduler-554b9d7496-p888c labeled
pod/stackstorm-ha-st2sensorcontainer-556d897b8c-7fsw7 labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-9hvr2 labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-xkzwq labeled
pod/stackstorm-ha-st2timersengine-79cd5b5c69-6rp5f labeled
pod/stackstorm-ha-st2web-7c96c574ff-2fd5n labeled
pod/stackstorm-ha-st2web-7c96c574ff-m4hh4 labeled
pod/stackstorm-ha-st2workflowengine-54dfb489b7-q4zkb labeled
pod/stackstorm-ha-st2workflowengine-54dfb489b7-wd247 labeled
job.batch/stackstorm-ha-job-st2-apikey-load labeled
job.batch/stackstorm-ha-job-st2-key-load labeled
job.batch/stackstorm-ha-job-st2-register-content labeled
deployment.apps/stackstorm-ha-st2actionrunner labeled
replicaset.apps/stackstorm-ha-st2actionrunner-844d5d96b9 labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-c7294 labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-czjff labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-nzzls labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-tmw2m labeled
pod/stackstorm-ha-st2actionrunner-844d5d96b9-w8c6x labeled
deployment.apps/stackstorm-ha-st2api labeled
replicaset.apps/stackstorm-ha-st2api-5499986674 labeled
pod/stackstorm-ha-st2api-5499986674-5kpgl labeled
pod/stackstorm-ha-st2api-5499986674-wqwrc labeled
deployment.apps/stackstorm-ha-st2auth labeled
replicaset.apps/stackstorm-ha-st2auth-7dfc9fdb8f labeled
pod/stackstorm-ha-st2auth-7dfc9fdb8f-mw5fp labeled
pod/stackstorm-ha-st2auth-7dfc9fdb8f-n8cld labeled
deployment.apps/stackstorm-ha-st2client labeled
replicaset.apps/stackstorm-ha-st2client-5bf8b694c8 labeled
pod/stackstorm-ha-st2client-5bf8b694c8-bmfb7 labeled
deployment.apps/stackstorm-ha-st2garbagecollector labeled
replicaset.apps/stackstorm-ha-st2garbagecollector-5f5964574d labeled
pod/stackstorm-ha-st2garbagecollector-5f5964574d-ksh8x labeled
deployment.apps/stackstorm-ha-st2notifier labeled
replicaset.apps/stackstorm-ha-st2notifier-5dc79694f7 labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-fnqkq labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-p7x4z labeled
deployment.apps/stackstorm-ha-st2rulesengine labeled
replicaset.apps/stackstorm-ha-st2rulesengine-6c57fbf4bd labeled
pod/stackstorm-ha-st2rulesengine-6c57fbf4bd-29678 labeled
pod/stackstorm-ha-st2rulesengine-6c57fbf4bd-6z8xh labeled
deployment.apps/stackstorm-ha-st2scheduler labeled
replicaset.apps/stackstorm-ha-st2scheduler-554b9d7496 labeled
pod/stackstorm-ha-st2scheduler-554b9d7496-4fzg5 labeled
pod/stackstorm-ha-st2scheduler-554b9d7496-p888c labeled
deployment.apps/stackstorm-ha-st2stream labeled
replicaset.apps/stackstorm-ha-st2stream-5bbd79dcd8 labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-9hvr2 labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-xkzwq labeled
deployment.apps/stackstorm-ha-st2timersengine labeled
replicaset.apps/stackstorm-ha-st2timersengine-79cd5b5c69 labeled
pod/stackstorm-ha-st2timersengine-79cd5b5c69-6rp5f labeled
deployment.apps/stackstorm-ha-st2web labeled
replicaset.apps/stackstorm-ha-st2web-7c96c574ff labeled
pod/stackstorm-ha-st2web-7c96c574ff-2fd5n labeled
pod/stackstorm-ha-st2web-7c96c574ff-m4hh4 labeled
deployment.apps/stackstorm-ha-st2workflowengine labeled
replicaset.apps/stackstorm-ha-st2workflowengine-54dfb489b7 labeled
pod/stackstorm-ha-st2workflowengine-54dfb489b7-q4zkb labeled
pod/stackstorm-ha-st2workflowengine-54dfb489b7-wd247 labeled
job.batch/stackstorm-ha-job-st2-apikey-load labeled
job.batch/stackstorm-ha-job-st2-key-load labeled
pod/stackstorm-ha-job-st2-apikey-load-d9w48 labeled
pod/stackstorm-ha-job-st2-key-load-bw8p7 labeled
job.batch/stackstorm-ha-job-st2-register-content labeled
pod/stackstorm-ha-job-st2-register-content-v2wwx labeled

After:

❯ k -n stackstorm get deployment/stackstorm-ha-st2actionrunner -o json | jq .metadata.labels
{
  "app": "st2actionrunner",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "st2actionrunner",
  "chart": "stackstorm-ha-0.100.0",
  "heritage": "Helm",
  "release": "stackstorm-ha",
  "tier": "backend",
  "vendor": "stackstorm"
}

arm4b
arm4b reviewed Jan 27, 2023
View reviewed changes
Copy link
Member

@arm4b arm4b left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

I think it makes sense to quickly tag a new v0.110.0 first based on current master allowing users to pin the version if needed before updating to this change.

@cognifloyd
Copy link
Member Author

cognifloyd commented Jan 27, 2023

Cutting another release first sounds good.

@cognifloyd
Copy link
Member Author

cognifloyd commented Jan 28, 2023

OK. Thank you for testing the migration script @mamercad! Seeing that output was very helpful. From the output, it looks like your ConfigMaps and Secrets did not get the app.kubernetes.io/name label. Would you please look at the labels in your test cluster and make sure everything (managed by this helm chart) has that label?

I tested the migration script with my test cluster and fixed a couple of issues with it:

  • improve the output so progress is clearer
  • handle the st2sensorcontainer deployments which can have generated app names
  • handle the extra-helm-hook jobs which also have generated app names

Note that re-running the script is safe, but ugly. If you re-run it you will see a bunch of errors you can ignore like:
'app.kubernetes.io/name' already has a value (st2workflowengine), and --overwrite is false
The script should continue past such errors, and we don't want to overwrite anything that is already set, so that's fine.

@cognifloyd cognifloyd marked this pull request as ready for review January 28, 2023 00:13
@cognifloyd
Copy link
Member Author

cognifloyd commented Jan 28, 2023

I'm marking this ready for review, but I won't merge it until after I cut the last 0-ver release.

@mamercad
Copy link
Contributor

mamercad commented Jan 28, 2023

OK. Thank you for testing the migration script @mamercad! Seeing that output was very helpful. From the output, it looks like your ConfigMaps and Secrets did not get the app.kubernetes.io/name label. Would you please look at the labels in your test cluster and make sure everything (managed by this helm chart) has that label?

I tested the migration script with my test cluster and fixed a couple of issues with it:

  • improve the output so progress is clearer
  • handle the st2sensorcontainer deployments which can have generated app names
  • handle the extra-helm-hook jobs which also have generated app names

Note that re-running the script is safe, but ugly. If you re-run it you will see a bunch of errors you can ignore like: 'app.kubernetes.io/name' already has a value (st2workflowengine), and --overwrite is false The script should continue past such errors, and we don't want to overwrite anything that is already set, so that's fine.

No worries, I blew everything away and started with master, then fetched and checked out your branch:

Output 
❯ helm ls -n stackstorm
NAME         	NAMESPACE 	REVISION	UPDATED                             	STATUS  	CHART                	APP VERSION
stackstorm-ha	stackstorm	1       	2023-01-27 19:23:30.599787 -0500 EST	deployed	stackstorm-ha-0.100.0	3.8
stackstorm-k8s  ≡

❯ git checkout cognifloyd/std-labels
Note: switching to 'cognifloyd/std-labels'.
...
HEAD is now at 658afdf Simplify migration script: the ServiceAccount token does not have labels.

❯ RELEASE_NAME=stackstorm-ha NAMESPACE=stackstorm migrations/standardize-labels.sh
Adding label app.kubernetes.io/instance=stackstorm-ha (which will replace release=stackstorm-ha) ...
configmap/stackstorm-ha-redis labeled
configmap/stackstorm-ha-redis-health labeled
configmap/stackstorm-ha-redis-scripts labeled
configmap/stackstorm-ha-st2-config labeled
configmap/stackstorm-ha-st2-pack-configs labeled
configmap/stackstorm-ha-st2-urls labeled
configmap/stackstorm-ha-st2actionrunner-post-start-script labeled
configmap/stackstorm-ha-st2client-post-start-script labeled
configmap/stackstorm-ha-st2tests labeled
secret/stackstorm-ha-rabbitmq-definitions labeled
secret/stackstorm-ha-st2-apikeys labeled
secret/stackstorm-ha-st2-auth labeled
secret/stackstorm-ha-st2-datastore-crypto-key labeled
secret/stackstorm-ha-st2-kv labeled
secret/stackstorm-ha-st2-ssh labeled
service/stackstorm-ha-redis labeled
service/stackstorm-ha-redis-headless labeled
service/stackstorm-ha-st2api labeled
service/stackstorm-ha-st2auth labeled
service/stackstorm-ha-st2stream labeled
service/stackstorm-ha-st2web labeled
serviceaccount/stackstorm-ha labeled
deployment.apps/stackstorm-ha-st2actionrunner labeled
deployment.apps/stackstorm-ha-st2api labeled
deployment.apps/stackstorm-ha-st2auth labeled
deployment.apps/stackstorm-ha-st2client labeled
deployment.apps/stackstorm-ha-st2garbagecollector labeled
deployment.apps/stackstorm-ha-st2notifier labeled
deployment.apps/stackstorm-ha-st2rulesengine labeled
deployment.apps/stackstorm-ha-st2scheduler labeled
deployment.apps/stackstorm-ha-st2sensorcontainer labeled
deployment.apps/stackstorm-ha-st2stream labeled
deployment.apps/stackstorm-ha-st2timersengine labeled
deployment.apps/stackstorm-ha-st2web labeled
deployment.apps/stackstorm-ha-st2workflowengine labeled
replicaset.apps/stackstorm-ha-st2actionrunner-6989dd88f4 labeled
replicaset.apps/stackstorm-ha-st2api-849bdff8c7 labeled
replicaset.apps/stackstorm-ha-st2auth-7f9494c759 labeled
replicaset.apps/stackstorm-ha-st2client-69f976747b labeled
replicaset.apps/stackstorm-ha-st2garbagecollector-5f5964574d labeled
replicaset.apps/stackstorm-ha-st2notifier-5dc79694f7 labeled
replicaset.apps/stackstorm-ha-st2rulesengine-79d568ff49 labeled
replicaset.apps/stackstorm-ha-st2scheduler-6696f84b6b labeled
replicaset.apps/stackstorm-ha-st2sensorcontainer-7fc9f89769 labeled
replicaset.apps/stackstorm-ha-st2stream-5bbd79dcd8 labeled
replicaset.apps/stackstorm-ha-st2timersengine-79cd5b5c69 labeled
replicaset.apps/stackstorm-ha-st2web-7c96c574ff labeled
replicaset.apps/stackstorm-ha-st2workflowengine-7b77749884 labeled
pod/stackstorm-ha-job-st2-apikey-load-9wdtz labeled
pod/stackstorm-ha-job-st2-key-load-96w6s labeled
pod/stackstorm-ha-job-st2-register-content-xn42q labeled
pod/stackstorm-ha-redis-node-0 labeled
pod/stackstorm-ha-redis-node-1 labeled
pod/stackstorm-ha-redis-node-2 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-h7fwb labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-kplx2 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-lzl2n labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-ntz6v labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-vf9h8 labeled
pod/stackstorm-ha-st2api-849bdff8c7-2grqq labeled
pod/stackstorm-ha-st2api-849bdff8c7-4g8bb labeled
pod/stackstorm-ha-st2auth-7f9494c759-5dwmv labeled
pod/stackstorm-ha-st2auth-7f9494c759-brg4r labeled
pod/stackstorm-ha-st2client-69f976747b-hfz8r labeled
pod/stackstorm-ha-st2garbagecollector-5f5964574d-w9mbx labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-4qqjz labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-ds5r2 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-hwjr9 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-nxhxt labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-qkslr labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-z6th8 labeled
pod/stackstorm-ha-st2sensorcontainer-7fc9f89769-78lvn labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-dhs7j labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-fhhpv labeled
pod/stackstorm-ha-st2timersengine-79cd5b5c69-xn5hc labeled
pod/stackstorm-ha-st2web-7c96c574ff-f62j2 labeled
pod/stackstorm-ha-st2web-7c96c574ff-w67gr labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-jrcz8 labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-t7zzk labeled
job.batch/stackstorm-ha-job-st2-apikey-load labeled
job.batch/stackstorm-ha-job-st2-key-load labeled
job.batch/stackstorm-ha-job-st2-register-content labeled

Adding label app.kubernetes.io/name=<app> (which will replace app=<app>) ...
configmap/stackstorm-ha-st2-config labeled
configmap/stackstorm-ha-st2-pack-configs labeled
configmap/stackstorm-ha-st2-urls labeled
configmap/stackstorm-ha-st2actionrunner-post-start-script labeled
configmap/stackstorm-ha-st2client-post-start-script labeled
secret/stackstorm-ha-rabbitmq-definitions labeled
secret/stackstorm-ha-st2-apikeys labeled
secret/stackstorm-ha-st2-auth labeled
secret/stackstorm-ha-st2-datastore-crypto-key labeled
secret/stackstorm-ha-st2-kv labeled
secret/stackstorm-ha-st2-ssh labeled
serviceaccount/stackstorm-ha labeled
Deployment app=st2actionrunner ...
deployment.apps/stackstorm-ha-st2actionrunner labeled
replicaset.apps/stackstorm-ha-st2actionrunner-6989dd88f4 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-h7fwb labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-kplx2 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-lzl2n labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-ntz6v labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-vf9h8 labeled
Deployment app=st2api ...
deployment.apps/stackstorm-ha-st2api labeled
replicaset.apps/stackstorm-ha-st2api-849bdff8c7 labeled
pod/stackstorm-ha-st2api-849bdff8c7-2grqq labeled
pod/stackstorm-ha-st2api-849bdff8c7-4g8bb labeled
Deployment app=st2auth ...
deployment.apps/stackstorm-ha-st2auth labeled
replicaset.apps/stackstorm-ha-st2auth-7f9494c759 labeled
pod/stackstorm-ha-st2auth-7f9494c759-5dwmv labeled
pod/stackstorm-ha-st2auth-7f9494c759-brg4r labeled
Deployment app=st2chatops ...
Deployment app=st2client ...
deployment.apps/stackstorm-ha-st2client labeled
replicaset.apps/stackstorm-ha-st2client-69f976747b labeled
pod/stackstorm-ha-st2client-69f976747b-hfz8r labeled
Deployment app=st2garbagecollector ...
deployment.apps/stackstorm-ha-st2garbagecollector labeled
replicaset.apps/stackstorm-ha-st2garbagecollector-5f5964574d labeled
pod/stackstorm-ha-st2garbagecollector-5f5964574d-w9mbx labeled
Deployment app=st2notifier ...
deployment.apps/stackstorm-ha-st2notifier labeled
replicaset.apps/stackstorm-ha-st2notifier-5dc79694f7 labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-4qqjz labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-ds5r2 labeled
Deployment app=st2rulesengine ...
deployment.apps/stackstorm-ha-st2rulesengine labeled
replicaset.apps/stackstorm-ha-st2rulesengine-79d568ff49 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-hwjr9 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-nxhxt labeled
Deployment app=st2scheduler ...
deployment.apps/stackstorm-ha-st2scheduler labeled
replicaset.apps/stackstorm-ha-st2scheduler-6696f84b6b labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-qkslr labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-z6th8 labeled
Deployment app=st2sensorcontainer ...
deployment.apps/stackstorm-ha-st2sensorcontainer labeled
replicaset.apps/stackstorm-ha-st2sensorcontainer-7fc9f89769 labeled
pod/stackstorm-ha-st2sensorcontainer-7fc9f89769-78lvn labeled
Deployment app=st2stream ...
deployment.apps/stackstorm-ha-st2stream labeled
replicaset.apps/stackstorm-ha-st2stream-5bbd79dcd8 labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-dhs7j labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-fhhpv labeled
Deployment app=st2timersengine ...
deployment.apps/stackstorm-ha-st2timersengine labeled
replicaset.apps/stackstorm-ha-st2timersengine-79cd5b5c69 labeled
pod/stackstorm-ha-st2timersengine-79cd5b5c69-xn5hc labeled
Deployment app=st2web ...
deployment.apps/stackstorm-ha-st2web labeled
replicaset.apps/stackstorm-ha-st2web-7c96c574ff labeled
pod/stackstorm-ha-st2web-7c96c574ff-f62j2 labeled
pod/stackstorm-ha-st2web-7c96c574ff-w67gr labeled
Deployment app=st2workflowengine ...
deployment.apps/stackstorm-ha-st2workflowengine labeled
replicaset.apps/stackstorm-ha-st2workflowengine-7b77749884 labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-jrcz8 labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-t7zzk labeled
Service app=st2api ...
service/stackstorm-ha-st2api labeled
Service app=st2auth ...
service/stackstorm-ha-st2auth labeled
Service app=st2chatops ...
Service app=st2stream ...
service/stackstorm-ha-st2stream labeled
Service app=st2web ...
service/stackstorm-ha-st2web labeled
Job app=st2 ...
job.batch/stackstorm-ha-job-st2-apikey-load labeled
job.batch/stackstorm-ha-job-st2-key-load labeled
pod/stackstorm-ha-job-st2-apikey-load-9wdtz labeled
pod/stackstorm-ha-job-st2-key-load-96w6s labeled
Job app=st2-apply-rbac-definitions ...
Job app=st2-register-content ...
job.batch/stackstorm-ha-job-st2-register-content labeled
pod/stackstorm-ha-job-st2-register-content-xn42q labeled
configmap/stackstorm-ha-st2tests labeled

❯ helm ls -n stackstorm
NAME         	NAMESPACE 	REVISION	UPDATED                             	STATUS  	CHART                	APP VERSION
stackstorm-ha	stackstorm	1       	2023-01-27 19:23:30.599787 -0500 EST	deployed	stackstorm-ha-0.100.0	3.8

❯ k -n stackstorm get deployment/stackstorm-ha-st2actionrunner -o json | jq .metadata.labels
{
  "app": "st2actionrunner",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "st2actionrunner",
  "chart": "stackstorm-ha-0.100.0",
  "heritage": "Helm",
  "release": "stackstorm-ha",
  "tier": "backend",
  "vendor": "stackstorm"
}

❯ k -n stackstorm get cm/stackstorm-ha-mongodb-scripts -o json | jq .metadata.labels
{
  "app.kubernetes.io/component": "mongodb",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "mongodb",
  "helm.sh/chart": "mongodb-10.0.1"
}

❯ k -n stackstorm get secrets stackstorm-ha-mongodb -o json | jq -r .metadata.labels
{
  "app.kubernetes.io/component": "mongodb",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "mongodb",
  "helm.sh/chart": "mongodb-10.0.1"
}

Let me know if there's anything else you want to see.

@cognifloyd cognifloyd requested review from arm4b and mamercad January 28, 2023 00:49
@mamercad
Copy link
Contributor

mamercad commented Jan 28, 2023

OK. Thank you for testing the migration script @mamercad! Seeing that output was very helpful. From the output, it looks like your ConfigMaps and Secrets did not get the app.kubernetes.io/name label. Would you please look at the labels in your test cluster and make sure everything (managed by this helm chart) has that label?
I tested the migration script with my test cluster and fixed a couple of issues with it:

  • improve the output so progress is clearer
  • handle the st2sensorcontainer deployments which can have generated app names
  • handle the extra-helm-hook jobs which also have generated app names

Note that re-running the script is safe, but ugly. If you re-run it you will see a bunch of errors you can ignore like: 'app.kubernetes.io/name' already has a value (st2workflowengine), and --overwrite is false The script should continue past such errors, and we don't want to overwrite anything that is already set, so that's fine.

No worries, I blew everything away and started with master, then fetched and checked out your branch:

Output
Let me know if there's anything else you want to see.

This actually has me thinking ... it wouldn't be too hard to write a workflow which ensures that all of the objects have the expected labeling.

@cognifloyd cognifloyd mentioned this pull request Jan 28, 2023
Closed
arm4b
arm4b reviewed Jan 28, 2023
View reviewed changes
@mamercad
Copy link
Contributor

mamercad commented Jan 28, 2023

OK. Thank you for testing the migration script @mamercad! Seeing that output was very helpful. From the output, it looks like your ConfigMaps and Secrets did not get the app.kubernetes.io/name label. Would you please look at the labels in your test cluster and make sure everything (managed by this helm chart) has that label?
I tested the migration script with my test cluster and fixed a couple of issues with it:

  • improve the output so progress is clearer
  • handle the st2sensorcontainer deployments which can have generated app names
  • handle the extra-helm-hook jobs which also have generated app names

Note that re-running the script is safe, but ugly. If you re-run it you will see a bunch of errors you can ignore like: 'app.kubernetes.io/name' already has a value (st2workflowengine), and --overwrite is false The script should continue past such errors, and we don't want to overwrite anything that is already set, so that's fine.

No worries, I blew everything away and started with master, then fetched and checked out your branch:
Output
Let me know if there's anything else you want to see.

This actually has me thinking ... it wouldn't be too hard to write a workflow which ensures that all of the objects have the expected labeling.

Oh, I completely missed the existence of the labels test unit test 🚀

mamercad
mamercad approved these changes Jan 28, 2023
View reviewed changes
Copy link
Contributor

@mamercad mamercad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@mamercad
Copy link
Contributor

mamercad commented Jan 28, 2023 •
edited
Loading

OK. Thank you for testing the migration script @mamercad! Seeing that output was very helpful. From the output, it looks like your ConfigMaps and Secrets did not get the app.kubernetes.io/name label. Would you please look at the labels in your test cluster and make sure everything (managed by this helm chart) has that label?
I tested the migration script with my test cluster and fixed a couple of issues with it:

  • improve the output so progress is clearer
  • handle the st2sensorcontainer deployments which can have generated app names
  • handle the extra-helm-hook jobs which also have generated app names

Note that re-running the script is safe, but ugly. If you re-run it you will see a bunch of errors you can ignore like: 'app.kubernetes.io/name' already has a value (st2workflowengine), and --overwrite is false The script should continue past such errors, and we don't want to overwrite anything that is already set, so that's fine.

No worries, I blew everything away and started with master, then fetched and checked out your branch:
Output
Let me know if there's anything else you want to see.

This actually has me thinking ... it wouldn't be too hard to write a workflow which ensures that all of the objects have the expected labeling.

Oh, I completely missed the existence of the labels test unit test 🚀

I played around adding "run the migration script" and "show the pod labels" afterwards (in the experimental E2E), seemingly can't direct link to steps, so, here's a couple of screenshots:

image

image

All I did was add these steps:

      - name: Run standardize labels migrations
        run: |
          ./migrations/standardize-labels.sh

      - name: Show the Pod labels
        run: |
          kubectl get pod -o json \
            | jq -r '.items[] | .metadata.labels'

Not quite sure where I'm going with this, just thinking about it ... here's the run.

@cognifloyd cognifloyd enabled auto-merge January 28, 2023 22:21
@cognifloyd cognifloyd disabled auto-merge January 28, 2023 22:30
@cognifloyd cognifloyd enabled auto-merge January 28, 2023 22:48
@cognifloyd cognifloyd merged commit eedb404 into StackStorm:master Jan 28, 2023
@cognifloyd cognifloyd mentioned this pull request Jan 28, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mamercad mamercad mamercad approved these changes

+1 more reviewer

@arm4b arm4b arm4b approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@cognifloyd cognifloyd

Labels

breaking change Helm K8s size/L PR that changes 100-499 lines. Requires some effort to review.

Projects

None yet

Milestone

v1.0.0

Development

Successfully merging this pull request may close these issues.

Migrate to Helm's recommended standard labels

3 participants

@cognifloyd @mamercad @arm4b